Skip to content

Correctness statistics for DSE#1587

Open
agusaldasoro wants to merge 12 commits into
dse/memoizationfrom
feat/correctness
Open

Correctness statistics for DSE#1587
agusaldasoro wants to merge 12 commits into
dse/memoizationfrom
feat/correctness

Conversation

@agusaldasoro

@agusaldasoro agusaldasoro commented Jun 16, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Adds measureDseCorrectness flag: after Z3 returns SAT, the generated rows are fed back through SqlHeuristicsCalculator to compute a distance to the original SQL predicate — logs a warning if
    distance is non-zero and records it in Statistics
  • Fixes composite primary key handling in SmtLibGenerator: the old code required every PK column to differ individually across rows, which was over-constrained (e.g. it ruled out (emp=1, proj=2) and (emp=1, proj=3) as a pair). Now emits an OR assertion so only the tuple must be distinct
  • Extends SmtLibGenerator to handle DELETE and UPDATE statements, not just SELECT
  • Fixes LongValue gene mapping: was calling .toInt() and silently truncating large values; now uses LongGene
  • Adds JSqlVisitor parsing improvements and new SmtLibGeneratorTest coverage

Motivation

The Z3 solver could generate rows that technically satisfy the SMT-LIB encoding but fail the original SQL predicate, due to encoding gaps (composite PKs, unsupported statement types, type
truncation). This PR adds a post-hoc verification step to detect those cases and tracks them as a metric, while also fixing the most common encoding bugs that caused them.

New Stats

CSV column Description
dseCorrectnessChecks Number of SAT results that were post-verified
dseCorrectnessZeroDistance Checks where generated rows fully satisfied the predicate
dseCorrectnessNonZero Checks where generated rows had non-zero distance to the predicate
dseCorrectnessAvgDist Average correctness distance across non-failure checks
dseCorrectnessEvalFailures Checks where the heuristic evaluator itself failed

@agusaldasoro agusaldasoro requested a review from jgaleotti June 16, 2026 01:45
@agusaldasoro agusaldasoro changed the title Add correctness statistics for DSE Correctness statistics for DSE Jun 18, 2026
@agusaldasoro agusaldasoro marked this pull request as ready for review June 18, 2026 11:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant